Méthodes multi-organes rapides avec a priori de forme pour la localisation et la segmentation en imagerie médicale 3D

With the ubiquity of imaging in medical applications (diagnostic, treatment follow-up, surgery planning. . . ), image processing algorithms have become of primary importance. Algorithms help clinicians extract critical information more quickly and more reliably from increasingly large and complex acquisitions. In this context, anatomy localization and segmentation is a crucial component in modern clinical workflows. Due to particularly high requirements in terms of robustness, accuracy and speed, designing such tools remains a challengingtask.In this work, we propose a complete pipeline for the segmentation of multiple organs in medical images. The method is generic, it can be applied to varying numbers of organs, on different imaging modalities. Our approach consists of three components: (i) an automatic localization algorithm, (ii) an automatic segmentation algorithm, (iii) a framework for interactive corrections. We present these components as a coherent processing chain, although each block could easily be used independently of the others. To fulfill clinical requirements, we focus on robust and efficient solutions. Our anatomy localization method is based on a cascade of Random Regression Forests (Cuingnet et al., 2012). One key originality of our work is the use of shape priors for each organ (thanks to probabilistic atlases). Combined with the evaluation of the trained regression forests, they result in shape-consistent confidence maps for each organ instead of simple bounding boxes. Our segmentation method extends the implicit template deformation framework of Mory et al. (2012) to multiple organs. The proposed formulation builds on the versatility of the original approach and introduces new non-overlapping constraintsand contrast-invariant forces. This makes our approach a fully automatic, robust and efficient method for the coherent segmentation of multiple structures. In the case of imperfect segmentation results, it is crucial to enable clinicians to correct them easily. We show that our automatic segmentation framework can be extended with simple user-driven constraints to allow for intuitive interactive corrections. We believe that this final component is key towards the applicability of our pipeline in actual clinical routine.Each of our algorithmic components has been evaluated on large clinical databases. We illustrate their use on CT, MRI and US data and present a user study gathering the feedback of medical imaging experts. The results demonstrate the interest in our method and its potential for clinical use.Avec l’utilisation de plus en plus répandue de l’imagerie dans la pratique médicale (diagnostic, suivi, planification d’intervention, etc.), le développement d’algorithmes d’analyse d’images est devenu primordial. Ces algorithmes permettent aux cliniciens d’analyser et d’interpréter plus facilement et plus rapidement des données de plus en plus complexes. Dans ce contexte, la localisation et la segmentation de structures anatomiques sont devenues des composants critiques dans les processus cliniques modernes. La conception de tels outils pour répondre aux exigences de robustesse, précision et rapidité demeure cependant un réel défi technique.Ce travail propose une méthode complète pour la segmentation de plusieurs organes dans des images médicales. Cette méthode, générique et pouvant être appliquée à un nombre varié de structures et dans différentes modalités d’imagerie, est constituée de trois composants : (i) un algorithme de localisation automatique, (ii) un algorithme de segmentation, (iii) un outil de correction interactive. Ces différentes parties peuvent s’enchaîner aisément pour former un outil complet et cohérent, mais peuvent aussi bien être utilisées indépendemment. L’accent a été mis sur des méthodes robustes et efficaces afin de répondre aux exigences cliniques. Notre méthode de localisation s’appuie sur une cascade de régression par forêts aléatoires (Cuingnet et al., 2012). Elle introduit l’utilisation d’informations a priori de forme, spécifiques à chaque organe (grâce à des atlas probabilistes) pour des résultats plus cohérents avec la réalité anatomique. Notre méthode de segmentation étend la méthode de segmentation par modèle implicite (Mory et al., 2012) à plusieurs modèles. La formulation proposée permet d’obtenir des déformations cohérentes, notamment en introduisant des contraintes de non recouvrement entre les modèles déformés. En s’appuyant sur des forces images polyvalentes, l’approche proposée se montre robuste et performante pour la segmentation de multiples structures. Toute méthode automatique n’est cependant jamais parfaite. Afin que le clinicien garde la main sur le résultat final, nous proposons d’enrichir la formulation précédente avec des contraintes fournies par l’utilisateur. Une optimisation localisée permet d’obtenir un outil facile à utiliser et au comportement intuitif. Ce dernier composant est crucial pour que notre outil soit réellement utilisable en pratique. Chacun de ces trois composants a été évalué sur plusieurs grandes bases de données cliniques (en tomodensitométrie, imagerie par résonance magnétique et ultrasons). Une étude avec des utilisateurs nous a aussi permis de recueillir des retours positifs de plusieurs experts en imagerie médicale. Les différents résultats présentés dans ce manuscrit montrent l’intérêt de notre méthode et son potentiel pour une utilisation clinique

Semantic Decomposition Improves Learning of Large Language Models on EHR Data

Electronic health records (EHR) are widely believed to hold a profusion of actionable insights, encrypted in an irregular, semi-structured format, amidst a loud noise background. To simplify learning patterns of health and disease, medical codes in EHR can be decomposed into semantic units connected by hierarchical graphs. Building on earlier synergy between Bidirectional Encoder Representations from Transformers (BERT) and Graph Attention Networks (GAT), we present H-BERT, which ingests complete graph tree expansions of hierarchical medical codes as opposed to only ingesting the leaves and pushes patient-level labels down to each visit. This methodology significantly improves prediction of patient membership in over 500 medical diagnosis classes as measured by aggregated AUC and APS, and creates distinct representations of patients in closely related but clinically distinct phenotypes.Comment: Extended Abstract presented at Machine Learning for Health (ML4H) symposium 2022, November 28th, 2022, New Orleans, United States & Virtual, http://www.ml4h.cc, 9 page

Multi-organ localization with cascaded global-to-local regression and shape prior

International audienceWe propose a method for fast, accurate and robust localization of several organs in medicalimages. We generalize global-to-local cascade of regression random forest to multiple organs. A firstregressor encodes global relationships between organs, learning simultaneously all organs parameters.Then subsequent regressors refine the localization of each organ locally and independently forimproved accuracy. We introduce confidence maps, which incorporate information about both theregression vote distribution and the organ shape through probabilistic atlases.They are used withinthe cascade itself, to better select the test voxels for the second set of regressors, and to provide richerinformation than the classical bounding boxes thanks to the shape prior.We propose an extensivestudy of the different learning and testing parameters, showing both their robustness to mediumvariations and their influence on the final algorithm accuracy.Finally we demonstrate the robustnessand accuracy of our approach by evaluating the localization of six abdominal organs (liver, twokidneys, spleen, gallbladder and stomach) on a large and diverse database of 130 CT volumes.Moreover, the comparison of our results with two existing methods shows significant improvementsbrought by our approach and our deep understanding and optimization of the parameters.</p

Multiple template deformation. Application to abdominal organ segmentation

International audienceWe propose a fast, automatic and versatile framework forthe segmentation of multiple anatomical structures from 2Dand 3D images. We extend the work of [1] on implicittemplate deformation to multiple targets. Our variational formulationoptimizes the non-rigid transformation of a set oftemplates according to image-derived forces. It embeds nonoverlappingconstraints ensuring a consistent segmentationresult. We demonstrate the potential of our approach on thesegmentation of abdominal organs (liver, kidneys, spleen andgallbladder) through an evaluation on 50 CT volumes. Ourmethod reaches state-of-the-art accuracy, ranging from 2mm(liver and kidneys) to 8mm (gallbladder).</p

Interactive Multi-Organ Segmentation based on Multiple Template Deformation

International audience<p>We present a new method for the segmentation of multipleorgans (2D or 3D) which enables user inputs for smart contour editing.By extending the work of [1] with user-provided hard constraints thatcan be optimized globally or locally, we propose an efficient and user-friendly solution that ensures consistent feedback to the user interactions.We demonstrate the potential of our approach through a user studywith 10 medical imaging experts, aiming at the correction of 4 organsegmentations in 10 CT volumes.We provide quantitative and qualitativeanalysis of the users' feedback.</p

Segmentation du corps calleux : un outil pronostique des pathologies inflammatoires cérébrales ?

National audienc

Deep Learning vs manual techniques for assessing left ventricular ejection fraction in 2D echocardiography: validation against CMR

International audienceStructured Abstract Objective To evaluate accuracy and reproducibility of 2D echocardiography (2DE) left ventricular (LV) volumes and ejection fraction (LVEF) estimates by Deep Learning (DL) vs. manual contouring and against CMR. Background 2DE LV manual segmentation for LV volumes and LVEF calculation is time consuming and operator dependent. Methods A DL-based convolutional network (DL1) was trained on 2DE data from centre A, then evaluated on 171 subjects with a wide range of cardiac conditions (49 healthy) – 31 subjects from centre A (18%) and 140 subjects from centre B (82%) – who underwent 2DE and CMR on the same day. Two senior (A 1 and B 1 ) and one junior (A 2 ) cardiologists manually contoured 2DE end-diastolic (ED) and end-systolic (ES) endocardial borders in the cycle and frames of their choice. Selected frames were automatically segmented by DL1 and two DL algorithms from the literature (DL2 and DL3), applied without adaptation to verify their generalizability to unseen data. Interobserver variability of DL was compared to manual contouring. All ESV, EDV and EF values were compared to CMR as reference. Results 50% of 2DE images were of good quality. Interobserver agreement was better by DL1 and DL2 than by manual contouring for EF (Lin’s concordance = 0.9 and 0.91 vs. 0.84), EDV (0.98 and 0.99 vs. 0.82), and ESV (0.99 and 0.99 vs. 0.89). LVEF bias was similar or reduced using DL1 (-0.1) vs. manual contouring (3.0), and worse for DL2 and DL3. Agreement between 2DE and CMR LVEF was similar or higher for DL1 vs. manual contouring (Cohen’s kappa = 0.65 vs. 0.61) and degraded for DL2 and DL3 (0.48 and 0.29). Conclusion DL contouring yielded accurate EF measurements and generalized well to unseen data, while reducing interobserver variability. This suggests that DL contouring may improve accuracy and reproducibility of 2DE LVEF in routine practice